Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

rate limit middleware to support long latency requests #10737

Closed
2 tasks done
inforly opened this issue May 18, 2024 · 1 comment
Closed
2 tasks done

rate limit middleware to support long latency requests #10737

inforly opened this issue May 18, 2024 · 1 comment
Labels
resolution/duplicate

Comments

@inforly
Copy link

inforly commented May 18, 2024

Welcome!

  • Yes, I've searched similar issues on GitHub and didn't find any.
  • Yes, I've searched similar issues on the Traefik community forum and didn't find any.

What did you expect to see?

rate limit middle ware can't work well with long latency requests, it should be due to the TTL. Since LLM (large language model) usually take long time (several to tens of seconds), it really needs rate limit to support such long latency requests. Please see the details of the issue: #10700
@inforly inforly changed the title rate limit support long latency requests rate limit middleware to support long latency requests May 18, 2024
@jspdown
Copy link
Contributor

jspdown commented May 23, 2024

Duplicate of #10700

@jspdown jspdown marked this as a duplicate of #10700 May 23, 2024
@jspdown jspdown closed this as completed May 23, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
resolution/duplicate
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@inforly @jspdown @traefiker